Block-based video compression standards such as H.261, H.263, MPEG1, MPEG2, and MPEG4 achieve efficient compression by reducing both temporal redundancies between video frames and spatial redundancies within a video frame. An intra-coded frame is self-contained and only reduces spatial redundancies within a video frame. Inter-coded frames, however, are predicted via motion compensation from previously coded frames to reduce temporal redundancies. The difference between the inter-coded video frame and its corresponding prediction is coded to reduce spatial redundancies. This methodology achieves high compression efficiency. However, the inter dependency between frames makes the coded bit-stream more susceptible to propagating channel errors. Errors introduced in the compressed bit-stream will result in errors in the reconstructed video frames. Due to the interdependent coding nature of a video frame, errors have the tendency of being propagated from one frame to another.
Any given macroblock (MB) within an inter-coded frame (i.e., Predicted frame (P-frame) or Bidirectionally predicted frame (B-frame)) may be coded as an intra macroblock. Similar to intra-coded frames, an intra macroblock is coded independently of data from a previously coded frame. The method of forcing macroblocks to be intra-coded is referred to as encoder intra-refresh. There are two main reasons for performing intra-refresh—inverse discrete cosine transform (IDCT) mismatch control and error resilience.
To control IDCT mismatch within the context of the H.261 and H.263 block-based video coding standards, each macroblock location in an image must be intra-coded at least once every 132 times that coefficients are transmitted for that macroblock. The intent of this is to limit the extent of error propagation due to DCT/IDCT mismatch. In other standards, the intra-coding rate is not specified.
To improve error resilience, selected macroblocks are forced to be intra-coded to limit error propagation resulting from using corrupt macroblocks that have been incorrectly reconstructed or concealed due to channel errors. These corrupt macroblocks may at times be visually objectionable. Furthermore, correctly decoded macroblocks from subsequent frames referencing back to a corrupt macroblock for temporal prediction may also be visually objectionable. These type of artifacts are typically more objectionable than the DCT/IDCT mismatch errors. As such, they drive the intra refresh strategy when communicating data over error-prone channels. A good intra-refresh strategy in the encoder will help limit error propagation in the decoder. This is done at the expense of generating more bits for intra-coding a macroblock.
Several methods have been disclosed in the prior art for determining intra-code refresh intervals. The different methods vary in both effectiveness/quality and computational complexity.
E. Steinbach, N. Farber, and B. Girod, “Standard Compatible Extension of H.263 for Robust Video Transmission in Mobile Environments,” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 7, No. 6, pp. 872-881, December 1997, discuss a back channel method for communicating corrupted group of blocks (GOBs) identified by the decoder to the encoder. In the context of existing video standards, a GOB is defined as a row of macroblocks. The encoder then tags these GOBs for intra-refreshing. Once the corresponding macroblocks have been intra-refreshed, the probability of limiting the error propagation increases. Although this is an effective method for intra-refreshing corrupt GOBs, it requires an active back channel in a two-way video communication application to be effective. Furthermore, there is a round trip delay introduced from when an error was injected into the bit-stream, detected by the decoder, and communicated to the encoder. There is an inherent propagation of error until the information is used by the encoder to intra-refresh the associated GOBs. Also, the back channel mechanism is rendered ineffective in video streaming applications where an existing encoded bit-stream is transmitted to a decoder.
J. Y Liao and J. Villasenor, “Adaptive Intra Block Update for Robust Transmission of H.263” IEEE Trans. On Circuits and Systems for Video Technology, Vol. 10, No. 1, pp. 30-35, February 2000, describe an adaptive intra-refresh strategy based on determining the sensitivity of a macroblock to errors. This error sensitivity metric is used to decide whether a macroblock should be intra-coded. One advantage of this method is that there is no need for a decoder to communicate information to an encoder over a back channel. A disadvantage of this approach is the added computational complexity introduced by the method. A statistical history of each macroblock is gathered and an error metric is computed based in part on its activity (i.e., number of bits generated), its location from a resynchronization marker, and the properties of the co-located macroblock from previously coded frames.
U.S. Pat. No. 6,025,888, issued on Feb. 15, 2000 to R. J. Pauls, entitled “Method and Apparatus for Improved Error Recovery in Video Transmission over Wireless Channels,” discloses a method for intra refreshing a macroblock based on the elapsed time since last intra coding of the macroblock position. A prescribed number of macroblocks with the longest elapsed time since last intra coding are forced intra-coded. The advantage of this method is that no back channel communication between a decoder and an encoder is necessary. However, this method requires keeping a running counter for each macroblock location to tag the macroblocks that should be intra-coded.
U.S. Pat. No. 5,491,509, issued on Feb. 13, 1996 to J. Jeong et al., entitled “Forced Intra-Frame Coding Method,” discloses a non-standards approach of having a vertical coding order of macroblocks. Lines of macroblocks spaced by a vertical interval are tagged for forced intra-coding. The intra-coded lines of macroblocks are shifted down one line from one frame to the next. The disadvantage of this approach is the non-standard compliance of the coding order. Another disadvantage of this approach is the large number of macroblocks that are forced intra-coded for each frame. This will generate more bits for the same video quality.
U.S. Pat. No. 5,724,369 issued on Mar. 3, 1998 to J. C. Brailean, K. J. O'Connell, M. R. Banham, and S. N. Levine, entitled “Method and Device for Concealment and Containment of Errors in a Macroblock-based Video Codec”, discloses an intra refresh method wherein the macroblock intra-coding order is defined by a number of scan lines. Similar to the previously disclosed techniques, one advantage of this method is that no back channel communication is required between a decoder and an encoder. Another advantage of this method is that it is a pre-determined intra-refresh strategy. Macroblocks are identified for intra-coding based on the frame identification (i.e. frame number or time instance). A major disadvantage of this technique is the regularity of the plurality of scan-lines. Since macroblock locations are intra-coded in a regular pattern from one frame to the next, a visually objectionable quality variation is observed. This regular intra refresh artifact detracts from the overall quality and content of a video sequence.
In light of the foregoing, there is an unmet need in the art to have a general-purpose, low complexity macroblock-based intra-refresh approach that takes advantage of the IDCT mismatch control and error resilience benefits inherent in intra-refresh while overcoming the problems with the prior art discussed above. This approach would preferably be applicable to arbitrary picture sizes without requiring corresponding hardware changes.